Floor and ceiling effects on the Montreal Cognitive Assessment in patients with Parkinson’s disease in Brazil

ABSTRACT Parkinson’s disease (PD) is a common neurodegenerative disease associated with cognitive impairment. The Montreal Cognitive Assessment (MoCA) has been used as a recommended global cognition scale for patients with PD, but there are some concerns about its application, partially due to the floor and ceiling effects. Objective: To explore the floor and ceiling effects on the MoCA in patients with PD in Brazil. Methods: Cross-sectional study with data from patients with PD from five Brazilian Movement Disorders Clinics, excluding individuals with a possible diagnosis of dementia. We analyzed the total score of the MoCA, as well as its seven cognitive domains. The floor and ceiling effects were evaluated for the total MoCA score and domains. Multivariate analyses were performed to detect factors associated with floor and ceiling effects. Results: We evaluated data from 366 patients with PD and approximately 19% of individuals had less than five years of education. For the total MoCA score, there was no floor or ceiling effect. There was a floor effect in the abstraction and delayed memory recall domains in 20% of our sample. The ceiling effect was demonstrated in all domains (80.8% more common in naming and 89% orientation), except delayed recall. Education was the main factor associated with the floor and ceiling effects, independent of region, sex, age at evaluation, and disease duration. Conclusion: The floor and ceiling effects are present in specific domains of the MoCA in Brazil, with a strong impact on education. Further adaptations of the MoCA structure for underrepresented populations may reduce these negative effects.


INTRODUCTION
P arkinson's disease (PD) is a neurodegenerative dis- ease with rising incidence and prevalence worldwide and is the fastest-growing neurological condition 1 .PD is characterized by progressive motor and non-motor symptoms, causing disability, negative impacts on quality of life, and elevated costs for public health systems 2 .Cognitive impairment is a common issue in PD, with approximately 23-55% of patients having mild cognitive impairment (MCI) 3 , which is a risk factor for dementia 3 .
The Mini-Mental State Examination (MMSE) is the most widely used cognitive screening instrument for elderly people.Unfortunately, MMSE presents some limitations in detecting MCI, probably due to the lack of tasks involving a more complex memory test, executive functions, higher-level language abilities, and visual-spatial processing 4 .The Montreal Cognitive Assessment (MoCA) was developed to increase the sensitivity to detecting MCI, including cognitive domains less tested in the MMSE 4 .Afterward, the MoCA was proposed as a superior tool for detecting cognitive impairment than the MMSE when applied in patients with PD 5 .A memory test with more words and a longer delay to recall and inclusion of executive functions and visual-spatial tasks supposedly converted the MoCA into a cognitive screening better fitted for PD than the MMSE 5 .Currently, the MoCA is a recommended global cognition scale for PD by the International Parkinson and Movement Disorder Society 6 .
However, studies with low-educated populations showed that the MoCA requires literacy abilities and is strongly influenced by age and education [7][8][9][10] .The cut-off score for cognitive decline varies across its numerous official language versions 11 , and the 1-point addition to the score for individuals with less than 13 years of education may not be sufficient as a correction factor for different cultural and educational settings 12 .
Floor and ceiling effects are present in a test when a substantial proportion of individuals register the minimum or maximum score, respectively, impairing the test's property to discriminate between subjects according to their true abilities 13 .Floor and ceiling effects may introduce bias in mean and variance estimates, which cause distortions in analytic methods.
The ceiling effect has been described for the MMSE, which impairs the discrimination of MCI in people with higher education and mild PD 5 .For the MoCA, the ceiling effect was not found in patients with PD 5 , but there is evidence of the floor effect in specific domains in these patients 14 .The present study aimed to explore the floor and ceiling effects on the MoCA in patients with PD in Brazil, a developing country with marked cultural diversity between regions.Furthermore, we evaluated if floor and ceiling effects were influenced by factors like education, regional diversity, sex, age, and disease duration.

Study design and ethical approval
We conducted a multicentric, observational, and cross-sectional study with data from Brazilian patients with PD to evaluate the floor and ceiling effects on the MoCA.We included patients followed at four Movement Disorders Clinics in Brazil (Ribeirão Preto, São Paulo, Porto Alegre, and Belém), which are part of the Latin American Research Consortium on the Genetics of Parkinson's Disease (LARGE-PD).Some patients from Ribeirão Preto were described in a previous study 14 .We also included patients from a fifth Movement Disorders Clinic (Brasília, Federal District).We enrolled patients between May 2007 and July 2022.The study was approved by the Ethics Committee from Hospital das Clínicas de Ribeirão Preto, and all participants provided written informed consent.

The Montreal Cognitive Assessment
The MoCA is a brief assessment that measures seven cognitive domains: visual-spatial/executive functions (0-5 points), naming (0-3 points), attention (0-6 points), language (0-3 points), abstraction (0-2 points), delayed memory recall (0-5 points), and orientation (0-6 points) 4 .The total MoCA score ranges from 0 to 30, with lower scores indicating inferior cognitive performance.We added one point to the total score when individuals had less than 13 years of education.There is a validated version of the MoCA for Brazilian Portuguese 17 .

Definitions of floor and ceiling effect
For the present study, we defined the floor effect when 15% or more of individuals from the sample achieved the score's worst level and the ceiling effect when 15% or more of individuals had the best score 18 .The floor and ceiling effects were evaluated for the total MoCA score and their domains.

Statistical analysis
To evaluate potential factors associated with floor and ceiling effects, we performed a multivariate logistic regression model and defined the presence of floor and ceiling as dependent variables.We selected site of origin, sex, age at evaluation, disease duration, and education (in years) as independent clinical variables.We used the Pearson's chi-square test to compare categorical variables and the Kruskal-Wallis test for continuous variables.All analyses were performed using Statistical Package for Social Sciences (SPSS) for Windows, version 23.0 (SPSS Inc., Chicago, USA), and graphical representations were generated using the R software version 4.0.4 and the R package ggplot2.

Clinical characteristics of the sample
We recruited 573 patients with PD from five Brazilian centers and excluded 127 for possible dementia and 80 due to missing data.A total of 366 patients were eligible for analysis (Table 1).There was a high clinical heterogeneity between participants from each of the five centers according to sex, disease duration, and education level (Table 1).Approximately 80% of them had mild disease (Hoehn & Yahr stages 1 and 2), and less than 7% were in advanced stages (Hoehn & Yahr stages 4 and 5).
Participants had a median of 11 years of education (high school level).The frequency of individuals with less than five years of formal education was 19.7%, with 3% of illiterate individuals (no formal education).In Brasilia, the median number of years of education (16 years) was two times higher than that of the other three centers (8 years; Ribeirão Preto, São Paulo, and Porto Alegre), with only 3.3% of individuals with less than five years of education.

Distribution of the total score and domains of the Montreal Cognitive Assessment between Brazilian centers
The distribution of the total score of the MoCA had a similar pattern in Belém, Ribeirão Preto, São Paulo, and Porto Alegre, with a peak density between 20-25 points (Table 1 and Figure 1).Brasília, the center with the highest frequency of individuals with education over 12 years, had a peak density close to 27 points.Regarding the domains of the MoCA, the scores varied significantly between centers and education levels (Table 2).

Floor and ceiling effect for the total score and domains of the Montreal Cognitive Assessment in patients with Parkinson's disease in Brazilian centers
For the total MoCA score, there was no ceiling effect (only 21 patients scored 30/30; 3.27%) in our sample, neither was detected floor effect due to the exclusion of patients with possible dementia.According to the domains of the MoCA, there was an overall floor effect in abstraction (20.2%) and delayed memory recall (19.3%) and an overall ceiling effect for all domains except delayed recall (10.3%), which was more common in naming (80.8%) and orientation (89%) (Figure 2).
Among centers, the floor effect in abstraction was detected in three of them (Belém 23.3%; Ribeirão Preto 31%; São Paulo 27%), as well as in delayed recall (Belém 28.8%; São Paulo 24.3%; Porto Alegre 20.4%).Brasilia was the unique center with a ceiling effect in all domains (Figure 2), probably due to the participants' high level of education.

Factors associated with floor and ceiling effect for domains of the Montreal Cognitive Assessment in patients with Parkinson's disease
After adjustment for the site of origin, sex, age at evaluation, and disease duration, the multivariate analysis showed that education was inversely associated with the floor effect for visual-spatial/executive functions, language, abstraction, delayed memory recall, and orientation (Table 3).Also, education was directly associated with the ceiling effect for all domains after adjustment, and female sex was associated with the floor effect for abstraction and with the ceiling effect for orientation (Table 3).

DISCUSSION
This multicentric study with patients with PD without dementia in Brazil showed that the floor effect in the    MoCA was present in abstraction and delayed memory recall domains in approximately 20% of our sample.
The ceiling effect was demonstrated in almost all evaluated domains, with the exception of delayed recall.These effects were not detected in the total score of the MoCA.Education was the main factor associated with the floor and ceiling effect in most domains, independent of region, sex, age at evaluation, and disease duration.
Only one-fifth of individuals had less than five years of formal education, including illiteracy.As described previously, our results showed a higher prevalence of the ceiling than floor effect in specific domains of the MoCA score 18 .The floor effect in abstraction and delayed memory recall domains was already described in Southeastern Brazil, but also in the subtopics of attention and language 14 .The authors suggest that emphasizing the instructions for patients and including adaptations for our population could reduce the number of individuals scoring zero.The same pattern was reported for the ceiling effect affecting all domains, except delayed memory.
Floor/ceiling effects on cognitive tests may cause adverse impacts because the measurement of part of the sample's true cognition (ability) is represented by censored data (observations partially known).Censoring reduces the capacity to highlight differences among lower-and higher-scoring individuals, concealing the proper range and variability of the measured cognitive function.Therefore, the exact levels of cognition are unknown due to these detrimental effects.Furthermore, floor/ceiling effects lead to a non-normal distribution of scores, artificial means and standard deviations, and weaken the reliability and validity of the analysis.Most used statistical tests are not adjusted for the censoring caused by floor/ceiling effects 13,19 .
Classically, floor/ceiling effects are described in total test scores, not in domains/subscales.As a neuropsychological battery, the MoCA may also be interpreted according to its specific domains.Compared to the MMSE, the MoCA was conceived to increase the detection of mild cognitive impairment, mainly based on visual-spatial/executive functions and delayed memory recall.Thus, it is reasonable to explore the presence of floor/ceiling effects in each domain.
Delayed memory recall is a crucial domain for neuropsychological batteries commonly used to diagnose cognitive decline in clinical practice.In the development of the MoCA, the authors proposed a memory test with more words, fewer learning trials, and a longer delay before recall than the MMSE 4 .According to our data, these adaptations may explain the floor effect in this domain.
In a sample of elderly individuals without PD from Tremembé, Brazil, with 68% of individuals having less than five years of education, the mean delayed memory recall subscores were lower than those seen in our results (no education: 0.8; 1-4 years: 0.94; 5-8 years: 1.79; 9-11 years: 1.6; ≥12 years: 2.13) 10 .In the Colombian city of Manizales, the authors also reported low mean delayed memory recall subscores in a sample with 49% of aged individuals without PD with less than five years This is the first study where the MoCA and its domains were evaluated in patients with PD from four Brazilian regions (North, Midwest, Southeast, and South).Brazil's socioeconomic, cultural, and linguistic diversity is represented in our sample, exploring the impact that cultural background may have on cognitive tests.Except for Brasilia, the regional diversity of included centers had a low impact on the MoCA total score and domains.We consider that the specific pattern of high scores and ceiling effects in all MoCA domains in Brasilia is probably due to the higher education level of its participants.Multivariate analyses showed no association between region heterogeneity and the presence of floor and ceiling effects.
Our sample had a low proportion of participants in the intermediate and advanced stages of disease (Hoehn & Yahr stages 3 to 5; 20.4%).Therefore, the impact of motor impairments on visual-spatial/executive functions domain-associated tasks (such as the trail-making test, copy of the cube, and clock drawing test) was probably limited.
Regarding the impact of education on the MoCA score, a recent study reinforced that high accuracy in discriminating normal individuals from people with dementia depends on the use of specific cutoff values stratified by education level 10 .For domains, our data from multivariate analyses showed a strong association of floor and ceiling effects with years of education, regardless of the socioeconomic and cultural diversity of Brazilian regions.In the MMSE, education is associated with floor/ceiling effects.The floor effect is frequent in people with low education, and the ceiling effect occurs mainly in highly educated individuals 20 .
Considering that approximately 16% of the global population may be illiterate and the low levels of education in developing countries, such as Brazil, the Mo-CA-Basic was designed to be applied to elderly adults in low-education settings, eliminating literacy-dependent tasks and adapting other tasks less dependent on education 21 .The MoCA-Basic scores did not differ according to educational level, without association with education or age 21 .There is no validation or normative data for the Brazilian Portuguese MoCA-Basic.
As a limitation, the absence of a formal cognitive status for all Brazilian centers did not assure the exclusion of all patients with dementia.Further, including individuals without PD would help understand the role of floor/ ceiling effects in people with PD and controls.Also, we had a small sample of people with less than five years of education compared to previous works in underrepresented regions 7,10 , which could have affected our analyses.The low number of participants with less than five years of education was also reported in other studies with MoCA in PD 22,23 , which may be related to a selection bias favoring high-educated people.The lack of a control group without PD and the low proportion of people with less than five years of education impair the generalizability of our findings on the floor and ceiling effects to a larger population.The strengths of our study were the large sample size and the socioeconomic and cultural diversity of individuals from different Brazilian regions.
In conclusion, the present study showed the presence of floor and ceiling effects in all domains of the MoCA, with a strong impact on education.Further adaptations of the MoCA structure, based on statistical methods such as item response theory, may reduce the floor/ceiling effects.

Figure 1 .
Figure 1.Distribution of the total score of the Montreal Cognitive Assessment in five Brazilian centers.

Floor and ceiling effects
on the Montreal Cognitive Assessment in patients with Parkinson's disease in Brazil Koshimoto et al.Dement Neuropsychol 2023;17:e20230022 Red bars over the red dotted line indicate the cutoff value for floor and ceiling effects.

Figure 2 .
Figure 2. Proportion of floor and ceiling effects on the Montreal Cognitive Assessment domains in five Brazilian centers.

Floor
and ceiling effects on the Montreal Cognitive Assessment in patients with Parkinson's disease in Brazil Koshimoto et al.Dement Neuropsychol 2023;17:e20230022

Table 1 .
Sample characterization and distribution of total score of the Montreal Cognitive Assessment of patients with Parkinson's disease in five Brazilian centers.

Table 2 .
Domains scores of the Montreal Cognitive Assessment in patients with Parkinson's disease from five Brazilian centers, by education level.